Introductionto Analysis Of Algorithms

Data Structures and Algorithms Made Easy Narasimha Karumanchi

Chapter 2 ANALYSIS OF ALGORITHMS

Introduction

This chapter will create the base for remaining all chapters. The objective of this chapter is to
tell you the importance of analysis of algorithms, their notations, relationships and solving as
many problems as possible. We first concentrate on understanding the importance of analysis
and then slowly move towards analyzing the algorithms with different notations. Finally, we
solve the problems. After completion of this chapter you will get the confidence in finding
the complexities of any given algorithm (especially recursive functions).

What is algorithm?

Just to understand better, let us consider the problem of preparing an omelet. For doing this,
the general steps which we follow are:

1) Get the frying pan.
2) Get the oil.
a. Do we have oil?
i. If yes, put it in the pan.
ii. If no, do we want to buy oil?
1. If yes, then go out and buy.
2. If no, we can terminate.
3) Turn on the stove.
4) Etc...

What actually we are doing is, for a given problem (in this case, preparing an omelet), we are
giving step by step procedure for solving it. So, the definition of algorithm can be given as:

An algorithm is the step-by-step instructions to a given problem.
step-by- problem.

66 Analysis of Algorithms | ©www.CareerMonk.com


One important thing while writing the algorithms is, we do not have to prove each step.
Also, all algorithms involve the basic elements which we have discussed in Introduction
chapter.

lgorithms?
Why analysis of algorithms?

Suppose if we want to go from city “A” to city “B”. There can be many ways of doing this: by
flight, by bus, by train and also by cycle. Depending on the availability and convenience we
choose the one which suits us. Similarly, in computer science there can be multiple
algorithms exist for solving the same problem. Algorithm Analysis helps us to determine
which of them is efficient in terms of time and space consumed.

Goal of analysis of algorithms?

The goal of analysis of algorithms is to compare algorithms (or solutions) mainly in terms of
running time but also in terms of other factors (e.g., memory, developer's effort etc.)

analysis?
What do we mean by running time analysis?

It’s the process of determining how processing time increases as the size of the problem
increases. Input size is number of elements in the input and depending on the problem type
the input may be of different types. Generally, we encounter the following input types.

• Size of an array
• Polynomial degree
• Number of elements in a matrix
• Number of bits in the binary representation of the input
• Vertices and edges in a graph

How do we compare algorithms?

In order to compare algorithms, we need to define some objective measures for comparing
algorithms. Now let us see based on what parameters we can compare the algorithms.

Can we compare using execution times?
This is not a good measure because times are specific to a particular computer.
Can we compare based on number of statements executed?



This is also not a good measure because the number of statements varies with the
programming language as well as the style of the individual programmer.
What is the Ideal Solution for comparing algorithms?
Suppose let us assume that we expressed running time of given algorithm as a function of the
input size n (i.e., f (n)). We can compare these different functions corresponding to running
times. This kind of comparison is independent of machine time, programming style, etc..

What is Rate of Growth?

The rate at which the running time increases as a function of input is called rate of growth.
Let us assume that we went to a shop for buying a car and a cycle. If your friend sees you
there and asks what you are buying then in general we say buying a car. This is because cost
of car is too big compared to cost of cycle. That means, we are approximating the cost of
cycle to cost of car.

Total Cost = cost_of_car + cost_of_cycle
Total Cost cost_of_car (approximation)

For the above example, we can represent the cost of car and cost of cycle in terms of function

large value of input size, n). As an example in the below case, , 2 , 100 and 500 are the
and for a given function we ignore the low order terms that are relatively insignificant (for

individual costs of some function and we approximate it to n4. Because is the highest rate
of growth.

2 100 500

Commonly used Rate of Growths

Below is the some list of rate of growths which we generally come across in the remaining
chapters.

Time complexity Name Example

O(1) Constant Adding an element to the front of a linked list

O(logn) Logarithmic Finding an element in a sorted array

O(n) Linear Finding an element in an unsorted array



O(nlogn) Linear Logarithmic Sorting n items by ‘divide-and-conquer’-Mergesort

O(n ) Quadratic Shortest path between two nodes in a graph

O(n ) Cubic Matrix Multiplication
O(2 ) Exponential The Towers of Hanoi problem

Below diagram shows the relationship between different rates of growth.

2

!
D
e
c

4
r
e
a

2
s
i
n
g

R

log
a

log !
t
e
s

O
f
2
G
r
o
w
t
h

log log

1



Types of Analysis

Let us assume that we have an algorithm for some problem and we want to know on what
inputs the algorithm is taking less time (performing well) and on what inputs the algorithm
is taking huge time.

Let us consider the following theory for better understanding. We have already seen that an
algorithm can be represented in the form of an expression. That means we represent
algorithm with multiple expressions: one for case where it is taking the less time and other
for case where it is taking the more time. In general the first case is called the best case for
the algorithm and second case is called the worst case for the algorithm.

So, to analyze an algorithm we need some kind of syntax and that forms the base for
asymptotic analysis/notation. There are three types of analysis:

• Worst case
o Defines the input on which the algorithm is taking huge time.
o Input is the one for which the algorithm runs the slower.
• Best case
o Defines the input on which the algorithm is taking lowest time.
o Input is the one for which the algorithm runs the fastest.
• Average case
o Provides a prediction about the running time of the algorithm
o Assumes that the input is random

Lower Bound Average Time Upper Bound

As an example, let us say f(n) is the function for a given algorithm. Also, assume the
following expression for best case and worst case.

f(n)= n + 500, for worst case
f(n)= n + 100n + 500 , for best case

Similarly, we can give expression for average case also. That means, the expression defines
the inputs with which the algorithm takes the average running time (or memory).

Asymptotic notation?



Once we have the expressions for best case, average case and worst case, we need to find the
upper bounds and lower bounds for each of them. That means, for all the three cases we need
to identify the upper bound, lower bounds. In order to represent these upper bound and
lower bounds we need some syntax and that creates the base for following discussion.

For the below discussions, let us assume that the given algorithm is represented in the form
of function .

Big-O notation
Big-

This notation gives the tight upper bound of the given function. Generally we represent it
as . That means, at larger values of n, the upper bound of is .

For example, if 100 10 50 is the given algorithm, then is .
That means gives the maximum rate of growth for at larger values of .

: there exist positive constants c and n such that 0
Now, let us see the O-notation with little more detail. O-notation defined as
for all
. is an asymptotic tight upper bound for . Our objective is to give some rate of
growth which is greater than given algorithms rate of growth .

values of is not important. In the below figure, n is the point from which we consider
In general, we do not consider lower values of . That means the rate of growth at lower

the rate of growths for a given algorithm. Below n the rate of growths may be different.
Also, for the remaining discussions we generally do not care for n . We directly give the
bound without bothering about these constants.

Rate of Growth cg(n)
f(n)

n Input Size, n
Big-
Big-O Visualization



O(1) O(n)
100,1000, 3n+100, 100,
200,1,20, etc. 100n, 2n-1, 3,
etc.
100N

n , 5n-10, 100,
O( O(nlogn)

n 2n 1, 5,
5nlogn, 3n-100,
2n-1, 100, 100n,
etc. etc.

. For example,
includes 1 , ,
is the set of functions with smaller or same order of growth as
etc..

values of only. That is the reason for having n in the definition. What this says is, below
Note: One important thing that we need to understand is we analyze the algorithms at larger

n we do not care for rates of growth. We follow the same for all notations.

Big-
Big-O examples

Below are few examples of Big-O notation.

Example-1 Find upper bound for f n
Example- 3n 8

Solution: 3 8 4 , for all n ≥ 8
∴3 8 = O(n) with c = 4 and n 8

Example- n 1

1 2 , for all n ≥ 1
∴ 1 = O( ) with c = 2 and n 1
Solution:

Example- n 100n 50

Solution: 100 50 2 , for all n ≥ 100



∴ 100 50 = O( ) with c = 2 and n 100

Example- n n

, for all n ≥ 1
∴ = O( ) with c = 1 and n 1
Solution:

Example- n

, for all n ≥ 1
∴ = O( ) with c = 1 and n 1
Solution:

Example- 100

Solution: 100 100, for all n ≥ 1
∴ 100 = O(1 ) with c = 1 and n 1

No uniqueness?

There is no unique set of values for n and c in proving the asymptotic bounds. Let us
consider, 100n 5 O . For this function there are multiple n and c.

Solution1: 100 5 100 101 101 5, n 5 and
101 is a solution.
for all

Solution2: 100 5 100 5 105 105 for all 1, 1 and
105 is also a solution.

Omega-
Omega- notation

Similar to above discussion, this notation gives the tighter lower bound of the given
algorithm and we represent it as Ω . That means, at larger values of n, the
tighter lower bound of f(n) is g(n).

For example, if 100 10 50 is the given algorithm, then is Ω .
That means, gives the tighter lower bound for at larger values of .

The Ω notation as be defined as Ω g n f n : there exist positive constants c and
n such that 0 cg n f n for all n n0 .
. Ω
is an asymptotic lower bound for
is the set of functions with smaller or same order of growth as .



Rate of Growth
f(n) cg(n)

n Input Size,
n
Examples

Below are few examples of Ω-Omega notation.

Example-
Example-1 Find lower bound for 5

Solution: ∃ , n Such that: 0 ≤ ≤ 5 ⇒ ≤ 5 1 and n = 1
∴5 1 and n = 1
⇒
Ω with

Example-
Example-2 Prove 100 5 Ω

Solution: ∃ , n Such that: 0 ≤ ≤ 100 5
100 5 ≤ 100 5 ∀ ≥1 105
≤ 105 ⇒ – 105 ≤ 0
Since is positive ⇒ – 105 ≤ 0 ⇒ ≤ 105/

Ω 2 , ,
⇒ Contradiction: cannot be smaller than a constant
1) Ω Ω

Theta-
Theta-θ notation

This notation decides whether the upper bound and lower bounds of a given function are
same or not. In simple terms, what we can say is, the average running time of growth is
always between lower bound and upper bound. If the upper bound (O) and lower bound (Ω)
gives the same result then θ notation will also have the same rate of growth.

As an example, let us assume that 10 is the expression. Then, its tight upper
bound is . The rate of growth in best case is . If we observe carefully,



the rate of growths in best case and worst are same. As a result the average case will also be
case.

Note: For a given function (algorithm), if the rate of growths (bounds) for O and Ω are not
same then the rate of growth θ case may not be same.

Now let us see the definition of θ notation. It is defined as Θ g n f n : there exist
positive constants c , c and n such that 0 c g n f n c g n for all n n 0 .
.
.
is an asymptotic tight bound for is the set of functions with the same
order of growth as

c g(n)
Rate of Growth

f(n)

c g(n)

n Input Size, n

θ Examples

Below are few examples of θ-Theta notation.
Example-
Example-1 Find θ bound for

1 and n = 1
Solution: , for all, n≥ 1
∴ θ with 1/5,

Example-
Example-2 Prove

Solution: ⇒ only holds for: 1/

∴
Example-3 Prove 6
Θ
Example- ≠



/6
∴6
Solution: n2 ≤ 6 ≤ ⇒ only holds for:
Θ

Example-
Example-4 Prove

Solution: log ⇒ ,∀ n0 – Impossible

Important Notes

From the above discussion it should be clear that, for each of the analysis (best case, worst
case and average) we try to give upper bound (O) and lower bound (Ω) and average running
time (θ). Also, from the above examples, it should also be clear that, for a given function
(algorithm) getting upper bound (O) and lower bound (Ω) and average running time (θ) may
not be possible always.

For example, if we are discussing the best case of an algorithm, then we try to give upper
bound (O) and lower bound (Ω) and average running time (θ).

In the remaining chapters we generally concentrate on upper bound (O) because knowing
lower bound (Ω) of an algorithm is of no practical importance. And, we use θ notation if
upper bound (O) and lower bound (Ω) are same.

Why we call Asymptotic Analysis?

From the above discussion (for all the three notations: worst case, best case and average case),
we can easily understand that, in every case we are trying to find other function g(n) for a
given function f(n) which approximates f(n) at higher values of n. That means, g(n) is also a
curve which approximates f(n) at higher values of n. In mathematics we call such curve as
asymptotic curve. In other terms, g(n) is the asymptotic curve for f(n). For this reason, we
call our analysis as asymptotic analysis.

Guidelines for asymptotic analysis?

There are some general rules to help us in determining the running time of an algorithm.
Below are few of them.

1) Loops The running time of a loop is, at most, the running time of the statements inside
Loops:
the loop (including tests) multiplied by the number of iterations.



// executes n times
for (i=1; i<=n; i++)
{
m = m + 2; // constant time, c
}

Total time = a constant c × n = c n = O(n)

2) Nested loops: Analyze inside out. Total running time is the product of the sizes of all the
loops:
loops.

//outer loop executed n times
for (i=1; i<=n; i++)
{
// inner loop executed n times
for (j=1; j<=n; j++)
{
k = k+1; //constant time
}
}

Total time = c × n × n = cn = O(n )

3) Consecutive statements: Add the time complexities of each statement.
statements:

x = x +1; //constant time

// executed n times
for (i=1; i<=n; i++)
{
m = m + 2; //constant time
}

//outer loop executed n times
for (i=1; i<=n; i++)
{
//inner loop executed n times
for (j=1; j<=n; j++)
{



k = k+1; //constant time
}
}

Total time = c + c n + c n2 = O (n )

4) If-then-else statements: Worst-case running time: the test, plus either the then part or
If-then- statements:
the else part (whichever is the larger).

//test: constant
if (length ( ) != otherStack. length ( ) )
{
return false; //then part: constant
}
else
{
// else part: (constant + constant) * n
for (int n = 0; n < length( ); n++)
{
// another if : constant + constant (no else part)
if (!list[n].equals(otherStack.list[n]))
//constant
return false;
}
}

Total time = c + c + (c +c ) * n = O(n)

5) Logarithmic complexity: An algorithm is O(log n) if it takes a constant time to cut the
complexity:
problem size by a fraction (usually by ½).

As an example let us consider the following program:

for (i=1; i<=n;)
{
i = i*2;
}

If we observe carefully, the value of i is doubling every time. That means, initially i =1, in
next step i =2, and in subsequent steps i = 4, 8 and so on.



Let us say the loop is executing some K times. That means at K-th step 2 n and we
come out of loop. So, if we take logarithm on both sides,

2
2
//if we assume base-2

So the total time = O(logn).

Note: Similarly, for the below case also the worst case rate of growth is O(logn). That means,
the same discussion holds good for decreasing sequence also.

for (i=n; i<=1;)
{
i = i/2;
}

Another example algorithm (binary search): finding a word in a dictionary of n pages
• Look at the centre point in the dictionary
• Is word to left or right of centre?
• Repeat process with left or right part of dictionary until the word is found

Properties of notations

Transitivity: f(n) = Θ(g(n)) and g(n) = Θ(h(n)) ⇒ f(n) = Θ(h(n)). Same for O and Ω.

Reflexivity: f(n) = Θ(f(n)). Same for O and Ω.

Symmetry: f(n) = Θ(g(n)) if and only if g n Θ f n .

Transpose symmetry: f(n) = O(g(n)) if and only if g(n) = Ω(f(n)).

Commonly used logarithms and summations

Below are the general mathematical relations that we use for solving some of our problems.
Remember we will not use these relations for all algorithms.

Logarithms



log ylog x
log n
log xy log x log y
n=
log log n = log(log n)
log = log x – log y

=

1
Arithmetic series

1 2
2

Geometric series

x2 …
n
k n
xn 1 1
1 x x 1
x 1
k 1

Harmonic series

1 1 1
1 … log
2

Other important formulae

log

1
1 2
1

Master Theorem for Divide and Conquer
All the efficient divide and conquer algorithms we will see divide the problems into
subproblems, each of which is some part of the original problem, and then perform some
additional work to compute the final answer. As an example, if we consider merge sort [in
detail, we will discuss in sorting chapter], it operates on two problems, each of which is half



the size of the original, and then uses additional work. This gives the running time
equation

2
2

The solution to this equation is . The following theorem can be used to determine
the running time of most divide and conquer algorithms. For a given program or algorithm,
first we try to find the recurrence relation for the problem. If the recurrence is of below form
then we directly give the answer without fully solving it.

The solution to the equation T n aT Θ nk , where a ≥1 and b > 1, is

O nlogb ,
a
bk
O nk logn ,
if a
T n bk
O nk ,
if a
if a bk

Variant of divide and conquer master theorem

The solution to the equation T n aT Θ nklog n , where a 1,b 1

O n , if a b
and p ≥ 0, is T n O n log n , if a b
O n log n , if a b

Problems on Master Theorem for Divide and Conquer

For each of the following recurrences, give an expression for the runtime T (n) if the
recurrence can be solved with the Master Theorem. Otherwise, indicate that the Master
Theorem does not apply.

Problem-1 T (n) = 3T (n/2)+ n
Solution:
Solution: T (n) = 3T (n/2)+ => T (n) = Θ( ) (Master Theorem Case 3)

Solution:
Solution: T (n) = 4T (n/2)+ => T (n) = Θ ( log n) (Master Theorem Case 2)



Problem-3 T (n) = T (n/2) + 2
Solution: T (n) = T (n/2) + 2 => Θ (2 ) (Master Theorem Case 3)
Solution:

Problem-4 T (n) = 2 T (n/2) + n
Solution: T (n) = 2 T (n/2) +
Solution: => Does not apply (a is not constant)

Solution:
Solution: T (n) = 16T (n/4)+ n => T (n) = Θ ( ) (Master Theorem Case 1)

Problem-6 T (n) = 2T (n/2)+ n log n
Solution:
Solution: T (n) = 2T (n/2)+ n log n => T (n) = n n (Master Theorem Case 2)

Problem-7 T (n) = 2T (n/2)+ n/ log n
Solution:
Solution: T (n) = 2T (n/2)+ n/ log n => Does not apply (non-polynomial difference between
f(n) and

Problem-8 T (n) = 2T (n/4)+ n .

. .
Solution:
Solution: T (n) = 2T (n/4)+ => T (n) = Θ ( ) (Master Theorem Case 3)

Problem-9 T (n) = 0.5T (n/2)+ 1/n
Solution:
Solution: T (n) = 0.5T (n/2)+ 1/n => Does not apply (a < 1)

Solution:
Solution: T (n) = 6T (n/3)+ log n => T (n) = Θ ( log n) (Master Theorem Case 3)

Problem-11 T (n) = 4T (n/2)+ n/ log n
Solution:
Solution: T (n) = 4T (n/2)+ n/ log n => T (n) = Θ ( ) (Master Theorem Case 1)

Problem-12 T (n) = 64T (n/8)− n log n
Solution:
Solution: T (n) = 64T (n/8)− log n => Does not apply (f(n) is not positive)



Solution:
Solution: T (n) = 7T (n/3)+ => T (n) = Θ ( ) (Master Theorem Case 3)

Problem-14 T (n) = 4T (n/2)+ log n
Solution:
Solution: T (n) = 4T (n/2)+ log n => T (n) = Θ ( ) (Master Theorem Case 1)

Problem-15 T (n) = T (n/2) + n(2 − cos n)
Solution:
Solution: T (n) = T (n/2) + n (2 − cos n) => Does not apply. We are in Case 3, but the
regularity condition is violated. (Consider n = 2Πk, where k is odd and arbitrarily large. For
any such choice of n, you can show that c >= 3/2, thereby violating the regularity condition.)

Problem-16 T (n) = 16T (n/4)+ n!
Solution:
Solution: T (n) = 16T (n/4)+ n! => T (n) = Θ (n!) (Master Theorem Case 3)

Problem-17 T (n) = √2T (n/2) + log n
Solution: T (n) = √2T (n/2) + log n => T (n) = Θ (√ ) (Master Theorem Case 1)
Solution:

Solution:
Solution: T (n) = 3T (n/2)+ n =>T (n) = Θ ( ) (Master Theorem Case 1)

Problem-19 T (n) = 3T (n/3)+ √n
Solution: T (n) = 3T (n/3)+ √ => T (n) = Θ (n) (Master Theorem Case 1)
Solution:

Problem-20 T (n) = 4T (n/2)+ cn
Solution: T (n) = 4T (n/2)+ cn => T (n) = Θ (
Solution: ) (Master Theorem Case 1)

Solution:
Solution: T (n) = 3T (n/4)+ n log n => T (n) = Θ (n log n) (Master Theorem Case 3)



Problem-22 T (n) = 3T (n/3)+ n/2
Solution:
Solution: T (n) = 3T (n/3)+ n/2 => T (n) = Θ (n log n) (Master Theorem Case 2)

Master Theorem for subtract and conquer recurrences

Let be a function defined on positive n, and having the property

c, if n 1
T n
aT n b f n , if n 1

for some constants c, a > 0, b > 0, k ≥ 0, and function f(n). If f(n) is in O(n ), then

O n , if a 1
T n O n , if a 1
O n a , if a 1

theorem
Variant of subtraction and conquer master theorem

The solution to the equation T n T αn T 1-α n β n, where 0 α 1 and β 0
are constants, is O nlogn .

Problems on Algorithms Analysis

Problem-23 Find the complexity of the below recurrence:
3 1 , 0,
1,
Solution: Let us try to solve this function with substitution.

T n 3T n 1
T n 3 3T n 2 3 T n 2
T n 3 3T n 3
.
.
.



T n 3 T n n 3 T 0 3
This clearly shows that the complexity of this function is 3 .

Note: We can directly use the Subtraction and Conquer master theorem for this problem.

Problem-24 Find the complexity of the below recurrence:
2 1 1, 0,
1,
Solution: Let us try to solve this function with substitution.
T n 2T n 1 1
T n 2 2T n 2 1 1 2 T n 2 2 1
T n 2 2T n 3 2 1 1 2 T n 4 2 2 2
T n 2 T n n 2 2 2 ….2 2 2
T n 2 2 2 2 ….2 2 2
T n 2 2 1 note: 2 2 2 2
T n 1
So the complexity is O 1 . Note that while the recurrence relation looks exponential the
solution to the recurrence relation here gives a different result.

Note: We can directly use the Subtraction and Conquer master theorem for this problem.

Problem-25 What is the running time of the following function (specified as a function of
the input value n)

void Function(int n)
{
int i=1 ;
int s=1 ;
while( s ≤ n)
{
i++ ;
s= s+i ;
print(*");
}
}



Solution:
Solution: Consider the comments in below function:

void Function (int n)
{
int i=1 ;
int s=1 ;
// s is increasing not at rate 1 but i
while( s ≤ n)
{
i++ ;
s= s+i ;
print(“*");
}
}

i. The value of ‘i’ increases
by one for each iteration. So the value contained in ‘s’ at the i iteration is the sum of the
We can define the terms ‘s’ according to the relation

first ‘i’ positive integers. If k is the total number of iterations taken by the program, then the
while loop terminates once

1
1 2 ... √ .
2

Problem-26 Find the complexity of the below function.

{
int i, count =0;;
for(i=1; i*i<=n; i++)
count++;
}


{
int i, count =0;;
for(i=1; i*i<=n; i++)
count++;
}



In the above function the loop will end, if √ . The reasoning is same
as that of Problem-25.

Problem-27 What is the complexity of the below program:

void function(int n)
{
int i, j, k , count =0;
for(i=n/2; i<=n; i++)
for(j=1; j<=n; j= j+ n/2)
for(k=1; k<=n; k= k * 2)
count++;
}

Solution:
Solution: Let us consider the comments in the following function.

{
//outer loop execute n/2 times
for(i=n/2; i<=n; i++)
//Middle loop executes n/2 times
for(j=1; j<=n; j= j+ n/2)
//outer loop execute logn times
for(k=1; k<=n; k= k * 2)
count++;
}

The complexity of the above function is O( ) only.

Problem-28 What is the complexity of the below program:

{
for(i=n/2; i<=n; i++)
for(j=1; j<=n; j= 2 * j)
for(k=1; k<=n; k= k * 2)
count++;



}

Solution:

{
//outer loop execute n/2 times
for(i=n/2; i<=n; i++)
//Middle loop executes logn times
for(j=1; j<=n; j= 2 * j)
//outer loop execute logn times
for(k=1; k<=n; k= k*2)
count++;
}

The complexity of the above function is O( ) only.

Problem-29 Find the complexity of the below program.

function( int n )
{
if ( n == 1 ) return ;
for( i = 1 ; i <= n ; i + + )
{
for( j= 1 ; j <= n ; j + + )
{
print(“*" ) ;
break;
}
}
}


function( int n )
{
//constant time
//outer loop execute n times



for( i = 1 ; i <= n ; i + + )
{
// inner loop executes only time due to break statement.
for( j= 1 ; j <= n ; j + + )
{
print(“*" ) ;
break;
}
}
}

The complexity of the above function is O(n) only. Even though the inner loop is bounded
by n, but due to the break statement it is executing only once.

Problem-30 Write a recursive function for the running time T(n) of the function function,
whose code is below. Prove using the iterative method that T(n) = Θ( ).
function( int n )
{
for( i = 1 ; i <= n ; i + + )
for( j = 1 ; j <= n ; j + + )
print(“*" ) ;
function( n-3 );
}

Solution:

function ( int n )
{
//constant time

//outer loop execute n times
for( i = 1 ; i <= n ; i + + )
//inner loop executes n times
for( j = 1 ; j <= n ; j + + )
//constant time
print(“*" ) ;

function( n-3 );



}

The recurrence for this code is clearly T(n) = T(n - 3) + c for some constant c > 0 since each
call prints out asterisks and calls itself recursively on n - 3. Using the iterative method we
get

T n T n 3 cn

Using the Subtraction and Conquer master theorem, we get T(n) = .

Problem-31 Determine Θ bounds for the following recurrence relation:

n
T n 2T nlogn
2

Solution: Using Divide and Conquer master theorem, we get O(n log n)
Solution:

Problem-32 Determine Θ bounds for the following recurrence

/2 /4 /8

Solution:
Solution: Substituting in the recurrence equation, we get:

1 2 3
2 4 8
, where k is a constant.


n n
T n 2T
2 logn

n n
T n 2T
Solution:
Solution:

2 logn
n n/2 n
2 2T
4 logn/2 logn
n n n
4T
4 logn/2 logn
n n n n n
2 T
2 logn/2 logn/2 logn/2 logn/2



n 1
2 T n
2 log n/2

We stop the expansion when 1, this implies that, k = logn.

n 1
T n 2 T n
2 log n/2
1
n n
log 2 /2
1
n n
log 2

Since log 2 k i log2 k i, the expression becomes:
1
n n
k i

// and are same. Its just matter of which comes irst.

1
n n
i
n nlogk
Θ nloglogn .


 /2 7

Solution: Using Master Theorem we get Θ lg n .
Solution:

Problem-35 Prove that the running time of the code below is (log n).

Read(int n);
{
int k = 1 ;
while( k < n )
k = 3k;
}



Solution:
Solution: The while loop will terminate once the value of ‘k’ is greater than or equal to the
value of ‘n’. Since each loop the value of ‘k’ is being multiplied by 3, if i is the number of

reaching i iterations when 3 ≥ n ↔ i ≥ log , which shows that i = (log n).
iterations, then ‘k’ has the value of 3i after i iterations. That is the loop is terminated upon

Problem-36 Use the recurrence tree to guess a good asymptotic bound for the following
recurrence relations and then prove Θ bounds:

T(n) = T(n − d) + T(d) + cn, where c > 0 and d ≥ 1 are constants.

Solution:
Solution:

For convenience we also define k = T(d), which we assume is some constant. We develop the
recursion tree below, labeling the levels using index i:

i=0 T(n) cn cn … cn

i=1 T(n-d) T(d) c(n-d) k c(n-d) k

i=2 T(n-2d) k c(n-2d) k
.
.
i= c(n-( )d) k

The recurrence for this can be written as:

1

1
2
2
2 2



For simplicity let us forget the constants c and d. Then the solution is Θ( ).

Problem-37 Solve the following recurrence.

1, 1
1 1 , 2

Solution: By iteration:

2 1 2 1

…

1 1

1

1 2 1 1
1
6 2
Ѳ

Problem-38 Consider the following program:

Fib[n]
if (n==0) then return 0
else if (n==1) then return 1
else return Fib[n-1]+Fib[n-2]

Solution: The recurrence relation for running time of this program is

1 2 .

Notice T n has two recurrence calls indicating a binary tree. Each step recursively calls the

number of leaves at depth n is 2 since this is a full binary tree, and each leaf takes at least
program for n only reduced by 1 and 2, so the depth of the recurrence tree is O(n). The

O(1) computation for the constant factor. So, the running time is clearly exponential in n.

Problem-39 Running time of following program?

function(n)



{
for( i = 1 ; i ≤ n ; i + + )
for( j = 1 ; j ≤ n ; j+ = i )
print( “*” ) ;
}

Solution:
function (n)
{
//this loop executes n times
for( i = 1 ; i ≤ n ; i + + )
//this loop executes j times with j increase by the rate of i
for( j = 1 ; j ≤ n ; j+ = i )
print( “*” ) ;
}

Its running time is √ since the inner loop is same as that of Problem-25].

Problem-40 What is the complexity of ∑ ?

Solution:
Solution: Using the logarithmic property that, log xy = log x + log y. We can see that this
problem is equivalent to

1 2

1 2 …
!

This shows that that the time complexity

Problem-41 What is the running time of the following recursive function (specified as a
function of the input value n)? First write a recurrence formula, and show its complexity.

function(int n)
{
if (n ≤ 1)



return ;
int i;
for (i=1 ; i ≤ 3; i++ )
f( );
}

olution:
function (int n)
{
//constant time
if (n ≤ 1)
return ;
int i;
//this loop executes with recursive loop of value
for (i=1 ; i ≤ 3; i++ )
f( );
}

recurrence for this code is T n 3T Θ 1 .
We can assume that for asymptotical analysis k = k = k for every integer k ≥1. The

Using master theorem, we get T(n) = Θ(n).

Problem-42 What is the running time of the following recursive function (specified as a
function of the input value n)? First write a recurrence formula, and show its solution
using induction.

function(int n)
{
if (n ≤ 1)
return;
for (i=1 ; i ≤ 3 ; i++ )
function (n − 1).
}

Solution:

function (int n)
{



//constant time
if (n ≤ 1)
return;

//this loop executes 3 times with recursive call of n-1 value
for (i=1 ; i ≤ 3 ; i++ )
function (n − 1).
}

The if statement requires constant time (call it c). With the for loop, we neglect the loop
overhead and only count the three times that the function is called recursively. This implies
a time complexity recurrence

, 1;
3 1 , 1.
Now we use repeated substitution to guess at the solution when we substitute k times:

3 1
Using the Subtraction and Conquer master theorem, we get T(n) = 3 .

Problem-43 Write a recursion formula for the running time of the function f, whose
code is below. What is the running time of f, as a function of ?

function (int n)
{
if (n ≤ 1)
return;
int i = 1 ;
for(i = 1; i < n; i + +)
print(“*”);
function ( 0.8n ) ;
}

Solution:

function (int n)
{
//constant time
if (n ≤ 1)
return;



//constant time
int i = 1 ;

// this loop executes times with constant time loop
for(i = 1; i < n; i + +)
print(“*”);

//recursive call with 0.8n
function ( 0.8n ) ;
}

.8
4
The recurrence for this piece of code is

5
4
5

Applying master theorem, we get T n O n .

Problem-44 Find the complexity of the following recurrence.

2 √

2
Solution: The given recurrence is not in the master theorem form. So we try to convert this
master theorem format. For that let use assume that

If we apply logarithm on both side, we get, logn mlog2 m logn

T n T 2 2T √2 m 2T 2 m.
Now, the given function becomes,

To make it simple we assume S m T 2 S T 2
S m 2S m

If we apply the master theorem then we get S m O m log m
If we substitute m log n back, T n S logn O logn log logn .


√ 1



Solution: We apply the same logic as that of Problem-44 and we get

m
S m S 1
2

If we apply the master theorem then we get S m O logm .

If we substitute m log n back, T n S log n O log logn .


2 √ 1
Solution: We apply the same logic as that of Problem-44 and we get

m
S m 2S 1
2

If we apply the master theorem then we get S m O m O m .
If we substitute m log n back, T n O logn .


int Function (int n)
{
if (n <= 2)
return 1;
else
return (Function (floor(sqrt(n))) + 1);
}


int Function (int n)
{
//constant time
if (n <= 2)
return 1;

// executes √ + 1 times
else

return (Function (floor(sqrt(n))) + 1);
}



For the above function, the recurrence function can be given as:

√ 1

And, this is same as that of Problem-45.

Problem-48 Analyze the running time of the following recursive procedure as a function of
n. You may assume that each assignment or division takes unit time.

{
if ( n < 2 )
return;
else
counter = 0;

for i = 1 to 8 do
function ( );
for I =1 to do
counter = counter + 1;
}


Let us refer to the running time of function (n) as T(n).

{
//constant time
if ( n < 2 )
return;
else
counter = 0;

// this loop executes 8 times with n value half in every call
for i = 1 to 8 do
function ( );

// this loop executes times with constant time loop



for I =1 to do
counter = counter + 1;
}
We have now defined T(n) as follows:

1 2,
n
8T 1 .
2

We can use the master theorem to solve this. log Ѳ log n .

temp = 1
repeat
for i = 1 to n

n = ;
temp = temp + 1;

until n ≤ 1

//const time
temp = 1
repeat
// this loops executes n times
for i = 1 to n

//recursive call with value
temp = temp + 1;

n = ;
until n ≤ 1

The recurrence for this function is
Using master theorem, we get, T(n) = O(n).

function(int n)
{
for( i = 1 ; i ≤ n ; i + + )
for( j = 1 ; j ≤ n ; j * = 2 )
print( “*” ) ;
}




function(int n)
{
// this loops executes n times
for( i = 1 ; i ≤ n ; i + + )
// this loops executes logn times from our logarithms
//guideline
for( j = 1 ; j ≤ n ; j * = 2 )
print( “*” ) ;
}

So the complexity of this program is : O(n logn).

function(int n)
{
for( i = 1 ; i ≤ n/3 ; i + + )
for( j = 1 ; j ≤ n ; j += 4 )
print( “*” ) ;
}


function(int n)
{
// this loops executes n/3 times
for( i = 1 ; i ≤ n ; i + + )
// this loops executes n-4 times
for( j = 1 ; j ≤ n ; j * = 2 )
print( “*” ) ;
}

So the complexity of this program is : O(n ).

{
if(n ≤ 1)



return;
if (n > 1)
{
print ("*");
function( );
function( );
}
}


{
//constant time
if(n ≤ 1)
return;
if (n > 1)
{
//constant time
print ("*");
//recursion with n/2 value
function( n/2 );
//recursion with n/2 value
function( n/2 );
}
}

n
T n 2T 1
The recurrence for this function is:

2
Using master theorem, we get T(n)=O(n).

Note: With these examples, I hope you got the idea of basic understanding of analysis. In this
Note:
chapter, try to understand when do we get different complexities like: O(logn), O(nlogn),
O(n), O(n ), O(2 ) etc… so that it will be clear when we refer them in remaining chapters.

Don’t worry. We will not use the same approach for the remaining chapters. We will just
directly give the complexity based on theory or based on master theorem.


Introductionto Analysis Of Algorithms

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (15)

Similar to Introductionto Analysis Of Algorithms

Similar to Introductionto Analysis Of Algorithms (20)

Recently uploaded

Recently uploaded (20)

Introductionto Analysis Of Algorithms